I have had a lot of lost data in my life. Take it from my AV, to my sensitive documents (which were pleasantly available physically), and my darn camera recently.
That's right. I had almost every photo I had taken on a single drive. Then, mid a file-transfer, I suddenly saw an IO error. I noticed my drive is not mounted anymore, and to make matters worse, when I took it out and reattached, my computer did not recognize it.
So, here I am, trying to write a public article as a reminder to myself, and a suggestion to others.
We live in a digital age. We handle a lot of files in our day to day life, some of them necessary, some of them very creative and fun, some of them nostalgic, unique, and rare to find. This begins the need for several items in our lives. We need our data available to us, and safe from predators, accidents, and passage of time.
There are different solutions to data storage
- Cloud Drives
- Thumb Drives (Flash Memory)
- Memory Cards (SD, micro-SD)
- External SSD
- External HDD
- CDs and DVD
Floppy
So, in this list, availability and reliability have a reverse relation. This is a not a professional opinion though, the list is not strict. The main issue with cloud drives' reliability is that someone is constantly charging you for availability of your data. And they might suddenly decide to stop the business and give you the middle finger. I loved CDs when they were more relevant, but now it's hard to find optical drivers on computers, and that's what makes them not very available.
By the way, in case you use cloud storage, check out rclone.
There are also solutions to data security.
- Unencrypted
- encrypted ZIP
- encrypted with PGP
- VeraCrypt (etc.)
- LUKS encrypted
Unencrypted is just it, there's no need for explanation. Do not leave any sensitive data unencrypted. This is very important. Your personal ID, passwords, keys, etc. should not be here. This is for consumed media. Your games, movies, series, etc.
Next in case of simplicity, we have zip files. I will include pgp files here, because even though they do seem scary, they are as easy if you learn and know how to use them. GPG also supports symmetric key encryption.
I have used encrypted ZIP files as well as GPG files in production. They are a hassle if a lot of cases, unless you do not want to edit them. For example, It is possible to add photos to a zip you're using like a folder to keep your files, but it is a hassle. Or if you want to add a few lines to a encrypted text file using GPG. (The second one is actually slightly usable in Emacs, but you would need a small setup.) Also note that I might have been using this wrong. However, I am almost sure ZIP files were never meant to encrypt a 32 GB folder of files.
We now get to other more recent and modern tools. VeraCrypt, and other famous third-party software like NordLocker. See more on Privacy Guides. Also, I have not used any of these tools, so I do not endorse them, nor do I reject them. I think they might be convenient. Neverthless, PG has a valid point on its "Minimum Qualifications" section. The main thing is, these software 1) make the interface easier, and 2) usually have a container like approach, similar to .zip files we talked about. They are not full disk encryption. This is both good and bad, be cautious.
So, full-disk encryption or not? I said yes. Now you see my failure, which was also caused by my lack of knowledge. FDE is useful in the sense that you do not have to worry about what is encrypted. If you mount the drive, you can do whatever you want. And then after ejection, things will be okay. However, it will make the drive slower, and every file slightly heavier. Contained approaches are good for this. When you have a lot of non sensitive data, with some sensitivity between them, you can encrypt a folder or a few files one by one instead.
LUKS is huge. It is the backbone of a lot of projects. Here we talk about the most basic of its use cases, FDE. As a small honorable mention that I dislike, but will be useful, Windows BitLocker and MacOS FileVault do exist. They're not open-source though. The main issue here is that if your drive fails and your encryption headers get corrupted, you will be in the mud. People suggest backing up your headers. This is from PG:
cryptsetup luksHeaderBackup /dev/device --header-backup-file /mnt/backup/file.img
And this is an arch-wiki page, and this is from void-linux.
Another very important thing to consider is backups. What is backup in
a nutshell? It's "copy your data in different places, so if you lose on,
you have another available". Simple. An easy way to do backups is to have
all your data on your internal storage in the laptop, as well as a external
storage at the same time. This can be easily achieved with an rsync. It
is also possible to clone your whole disk to another disk, I have not used
these tools though. In case of more advanced setups, you can always opt for
different RAID setups and also search around for NAS. These are also out of
the scope of my knowledge and experience, but TLDR, RAID is a system to use
multiple drives as one drive, and NAS is a small storage server to share
data over network.
Now that we are done doing a overview, let me suggest a workflow. I think almost any files you consider document should be encrypted as well as all of your personal pictures and images. I have lost data on both failing memory cards and flash memories, so I think the best solution would be SSD or HDD. If the files are not too large, CD are also a very great way of preserving data. Unfortunately, we live in a day and age that companies take away any of your data they dislike, so you have to take care of even your nostalgic movies that you liked in your childhood.